Method for receiving a broadcast stream and detecting and classifying direct response advertisements using fingerprints

ABSTRACT

System and method for detecting and classifying direct response advertisements. The system includes a unit for generating an advertisement candidate segment for an advertisement section detected from a broadcast stream; a matching unit for determining whether the candidate segment matches each advertisement segment stored in a database (DB); a unit for, if the matching unit determines that a segment matching the candidate segment is not present, determining whether the candidate segment is a direct response advertisement; a registration unit for storing the candidate segment, determined to be a direct response advertisement, as an advertisement segment that is the direct response advertisement in the DB; and a direct response advertisement grouping unit for, if the matching unit determines that an advertisement segment matching the candidate segment is present, and the matching segment is a direct response advertisement, grouping the candidate segment with DB-stored advertisement segments that are direct response advertisements.

CROSS-REFERENCE TO THE RELATED APPLICATIONS

This is a continuation of application Ser. No. 14/740,234, filed Jun.15, 2015, which is, in turn, a continuation of application Ser. No.14/117,369, filed Nov. 12, 2013, which is, in turn, a national stage ofapplication number PCT/KR2013/007475, filed Aug. 20, 2013. Furthermore,this application claims also the benefit of priority of Koreanapplication number 10-2013-0093744, filed Aug. 7, 2013. The disclosuresof these prior applications are incorporated herein by reference.

TECHNICAL FIELD

The present invention relates, in general, to a system and method fordetecting and classifying direct response advertisements (DR ad) and,more particularly, to a system and method for detecting and classifyingdirect response advertisements, which can detect direct responseadvertisements provided via broadcasts and group the detected directresponse advertisements by identicalness.

BACKGROUND ART

Some television (TV) commercial broadcasts use a scheme for exposinginformation, such as a phone number, an Internet homepage address, and aprice, at a specific location of an advertisement image for apredetermined period of time in order for a viewer or a user to requestconsultation about an advertised product or to order the correspondingproduct. Such a type of advertising is referred to as “Direct ResponseAdvertising (DR ad)”.

Some of such DR advertisements are delivered after pieces of informationabout a phone number, an Internet homepage address, and a price havebeen changed so that, for the same commercial broadcast content, DRadvertisements are suitable for the corresponding areas of individualcountries or individual areas of a specific country. If, with respect tosuch changed advertisements produced from the same advertisement,statistical information such as the number of times the correspondingchanged advertisement is delivered for each changed form can beprovided, this statistical information is very useful.

Meanwhile, as methods of detecting an advertisement when anadvertisement is included in a broadcast, technology such as thatdisclosed in the paper in the following non-patent document is known inthe art.

Further, Korean Patent Application Publication No. 10-2011-0129325 (Dateof Publication: Dec. 1, 2011) relates to an “Advertisement imagedetection method and apparatus” and discloses an advertisement imagedetection method including the steps of dividing each of a plurality offrames of a monitoring target image in which broadcast content and anadvertisement image played on an image play device are mixed intopartial blocks having a predetermined size; calculating pieces offeature information about respective partial blocks corresponding to apreset location among the partial blocks having the predetermined size;converting the pieces of feature information into a feature informationvector; and comparing a feature information vector of a referenceadvertisement image desired to be detected, which is previously storedin a database (DB), with the feature information vector, thus detectingthe reference advertisement image.

However, all of these technologies merely determine whether anadvertisement previously registered in the DB is being played duringbroadcasting, and thus there are limitations in that it is impossible todetect direct response advertisements that induce the direct response ofa user and it is also impossible to group direct response advertisementsdepending on their changed forms.

PRIOR ART DOCUMENTS Patent Documents

-   Korean Patent Application Publication No. 10-2011-0129325 (Date of    Publication: Dec. 1, 2011)

Non-Patent Documents

-   D. A. Sadlier et al., “Automatic TV advertisement detection from    mpeg bitstream,” Journal of the Patt. Rec. Society, vol. 35, no. 12,    pp. 215, December 2002.

DISCLOSURE Technical Problem

Accordingly, the present invention has been made keeping in mind theabove limitations, and an object of the present invention is to providea system and method which can determine whether the correspondingadvertisement section corresponds to a direct response advertisement ifthe advertisement section is detected from a broadcast stream.

Another object of the present invention is to provide a system andmethod which can group advertisements determined to be direct responseadvertisements by identicalness.

A further object of the present invention is to provide a system andmethod which can group direct response advertisements depending on theirchanged forms, so that various types of statistical information andadditional information can be provided for respective changed forms,thus exactly evaluating the efficiency of advertising related to directresponse advertising.

Technical Solution

In order to accomplish the above objects, the present invention providesa system for detecting and classifying direct response advertisements,including an advertisement candidate segment generation unit forgenerating an advertisement candidate segment for an advertisementsection detected from a broadcast stream; a matching unit fordetermining whether the advertisement candidate segment matches eachadvertisement segment stored in an advertisement database (DB); a directresponse advertisement determination unit for, if it is determined bythe matching unit that an advertisement segment matching theadvertisement candidate segment is not present, determining whether theadvertisement candidate segment is a direct response advertisement; adirect response advertisement registration unit for storing theadvertisement candidate segment, determined to be the direct responseadvertisement by the direct response advertisement determination unit,as an advertisement segment that is the direct response advertisement inthe advertisement DB; and a direct response advertisement grouping unitfor, if it is determined by the matching unit that an advertisementsegment matching the advertisement candidate segment is present, and thematching advertisement segment is a direct response advertisement,grouping the advertisement candidate segment with advertisement segmentsthat are direct response advertisements stored in the advertisement DB.

In this case, the direct response advertisement determination unit maybe configured to, if an advertisement corresponding to the advertisementcandidate segment is a video signal, determine whether the advertisementcandidate segment is a direct response advertisement by checking whethera pre-designated character string indicating a direct responseadvertisement is included in at least one of image frames constitutingthe advertisement video signal.

Further, the direct response advertisement determination unit mayacquire one or more pieces of region information related to a locationat which the pre-designated character string is displayed in an imageframe, and the direct response advertisement registration unit may storethe region information, information about the character string, andinformation about reliability of checking the character string asadvertisement segment information for the advertisement segment in theadvertisement DB.

Furthermore, the direct response advertisement determination unit may beconfigured to, if an advertisement corresponding to the advertisementcandidate segment is an audio signal, determine whether theadvertisement candidate segment is a direct response advertisement bychecking whether a pre-designated audio signal indicating a directresponse advertisement is included in the advertisement audio signal.

Furthermore, the direct response advertisement determination unit mayacquire one or more pieces of time information related to a location atwhich the pre-designated audio signal appears, and the direct responseadvertisement registration unit may store the time information,information about the pre-designated audio signal, and information aboutreliability of checking the audio signal information as advertisementsegment information for the advertisement segment in the advertisementDB.

Furthermore, the direct response advertisement registration unit may setthe advertisement segment stored in the advertisement DB to a mastersegment of the corresponding direct response advertisement.

Furthermore, the direct response advertisement grouping unit maydetermine whether the advertisement candidate segment is identical to anadvertisement segment that is a master segment of the advertisementsegment matching the advertisement candidate segment by comparing thosesegments with each other, and if, as a result of the determination ofidenticalness, the advertisement candidate segment is identical to theadvertisement segment that is the master segment, the direct responseadvertisement grouping unit may set the advertisement candidate segmentas an instance segment of the master segment, whereas if theadvertisement candidate segment is not identical to the advertisementsegment that is the master segment, the direct response advertisementgrouping unit may set the advertisement candidate segment to a variationsegment of the master segment.

Furthermore, the direct response advertisement grouping unit maydetermine whether the advertisement candidate segment is identical to anadvertisement segment that is a master segment of the advertisementsegment matching the advertisement candidate segment by comparing thosesegments with each other, if, as a result of the determination ofidenticalness, the advertisement candidate segment is identical to theadvertisement segment that is the master segment, the direct responseadvertisement grouping unit may set the advertisement candidate segmentto an instance segment of the master segment, and if the advertisementcandidate segment is not identical to the advertisement segment that isthe master segment, the direct response advertisement grouping unit maybe configured to determine whether the advertisement candidate segmentis identical to an advertisement segment that is a variation segment ofthe master segment by comparing those segments with each other, and isconfigured to, if it is determined that the advertisement candidatesegment is identical to the advertisement segment that is the variationsegment, set the advertisement candidate segment to an instance segmentof the variation segment, whereas if it is determined that theadvertisement candidate segment is not identical to the advertisementthat is the variation segment, set the advertisement candidate segmentto a new variation segment of the master segment.

Furthermore, determination of identicalness may be performed bycomparing an image of comparing an image of a region corresponding toregion information at which the character string, indicating a directresponse advertisement and included in the advertisement candidatesegment, is located with an image of a region corresponding to regioninformation of a character string for an advertisement segment that is acomparison target.

Furthermore, comparison of the images may be performed based on resultsof calculation of a distance function between images that are comparisontargets.

Furthermore, determination of identicalness may be performed bycomparing character string information, which indicates a directresponse advertisement and is included in the advertisement candidatesegment, with character string information stored for an advertisementsegment that is a comparison target.

In accordance with another aspect of the present invention, there isprovided a method of detecting and classifying direct responseadvertisements, including a first step of generating an advertisementcandidate segment for an advertisement section detected from a broadcaststream; a second step of determining whether the advertisement candidatesegment matches each advertisement segment stored in an advertisementdatabase (DB); a third step of, if it is determined at the second stepthat an advertisement segment matching the advertisement candidatesegment is not present, determining whether the advertisement candidatesegment is a direct response advertisement; a fourth step of storing theadvertisement candidate segment, determined to be the direct responseadvertisement at the third step, as an advertisement segment that is thedirect response advertisement in the advertisement DB; and a fifth stepof, if it is determined at the second step that an advertisement segmentmatching the advertisement candidate segment is present, and thematching advertisement segment is a direct response advertisement,grouping the advertisement candidate segment with advertisement segmentsthat are direct response advertisements stored in the advertisement DB.

Advantageous Effects

In accordance with the present invention, there can be provided a systemand method, which can determine whether the corresponding advertisementsection corresponds to a direct response advertisement if theadvertisement section is detected from a broadcast stream.

Further, in accordance with the present invention, there can be provideda system and method, which can group advertisements determined to bedirect response advertisements by identicalness.

Furthermore, in accordance with the present invention, there is theadvantage of providing a system and method, which can group directresponse advertisements depending on their changed forms, so thatvarious types of statistical information and additional information canbe provided for respective changed forms, thus exactly evaluating theefficiency of advertising related to direct response advertising.

DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing the configuration of an embodiment of asystem 100 for detecting and classifying direct response advertisementsaccording to the present invention;

FIG. 2 is a diagram showing a procedure for acquiring regioninformation;

FIG. 3 is a diagram showing the concept of a grouping procedureperformed by a direct response advertisement grouping unit 50;

FIG. 4 illustrates a master segment and variation segments, and instancesegments of the master and each variation segment, such as thosedescribed in FIG. 3, together with actual screens;

FIG. 5 is a diagram showing a procedure for determining identicalnessbetween pieces of region information;

FIG. 6 is a diagram illustrating a procedure for setting a variationsegment;

FIG. 7 is a diagram showing an example of advertisement segmentinformation stored in an advertisement database (DB) 60 used in thesystem 100 of the present invention;

FIG. 8 is a flowchart showing an embodiment of a method performed by thesystem 100 for detecting and classifying direct response advertisementsaccording to the present invention, which has been described withreference to FIGS. 1 to 7; and

FIG. 9 is a detailed flowchart showing a grouping procedure at stepS150.

BEST MODE

Hereinafter, embodiments of the present invention will be described indetail with reference to the attached drawings.

FIG. 1 is a diagram showing the configuration of an embodiment of asystem 100 for detecting and classifying direct response advertisementsaccording to the present invention (hereinafter referred to as a “system100”).

Referring to FIG. 1, the system 100 includes an advertisement candidatesegment generation unit 10, a matching unit 20, a direct responseadvertisement determination unit 30, a direct response advertisementregistration unit 40, and a direct response advertisement grouping unit50. The system 100 further includes an advertisement database (DB) 60.

The advertisement candidate segment generation unit 10 functions togenerate an advertisement candidate segment for an advertisement sectiondetected from a broadcast stream. The term “broadcast stream” denotesbroadcast signals provided via, for example, terrestrial TV broadcastsor cable TV broadcasts. The advertisement candidate segment generationunit 10 detects whether an advertisement is included in a broadcaststream, that is, whether an advertisement section is included in thebroadcast stream, while receiving the broadcast stream.

As technology for detecting an advertisement section from a broadcaststream, any type of technology well known in the art may be used, andthe detection of an advertisement section is not a direct object of thepresent invention, and thus a detailed description thereof will beomitted. Meanwhile, Korean Patent Application No. 10-2013-0068305 filedby the present applicant relates to “Advertisement detection system andmethod based on fingerprints”, and the advertisement detection methoddisclosed in this patent may also be used.

The advertisement candidate segment generation unit 10 detects anadvertisement section, and generates an advertisement candidate segmentfor the detected advertisement section, wherein the advertisementcandidate segment may be generated by dividing the advertisement sectioninto units of a predetermined time length. Further, the advertisementcandidate segment generation unit 10 acquires advertisement candidatesegment information about the generated advertisement candidate segment.

The advertisement candidate segment information may include informationabout a broadcast channel for providing a broadcast stream from whichthe advertisement section is detected, information about the time atwhich the broadcast stream from which the advertisement section isdetected is provided, and time information about the advertisementsection. Further, the advertisement candidate segment information mayinclude fingerprint information for at least a part of the advertisementsection. Further, when the detected advertisement is provided as avideo, the advertisement candidate segment information may includesource digital data of the corresponding advertisement video. When thedetected advertisement is provided as audio, the advertisement candidatesegment information may include source digital data of the correspondingadvertisement audio.

In this way, the advertisement candidate segment generation unit 10acquires advertisement candidate segment information while generatingthe advertisement candidate segment.

The matching unit 20 functions to determine whether the advertisementcandidate segment generated by the advertisement candidate segmentgeneration unit 10 matches each advertisement segment stored in theadvertisement DB 60. The determination of matching/non-matching may beimplemented by comparing the fingerprint information of theadvertisement candidate segment generated by the above-describedadvertisement candidate segment generation unit 10 with the fingerprintinformation of the advertisement segment stored in the advertisement DB60. Technology for performing matching by comparing pieces offingerprint information is not a direct object of the present invention,and configurations known in the conventional technology may be usedwithout change, and thus a detailed description thereof will be omitted.

For this, the advertisement DB 60 stores the fingerprint information ofeach advertisement segment as advertisement segment information. In thiscase, the term “advertisement segment” denotes an advertisementcandidate segment registered as a direct response advertisement or as anormal advertisement in the advertisement DB via a predeterminedprocedure, which will be described later, using the direct responseadvertisement determination unit 30, the direct response advertisementregistration unit 40, and the direct response advertisement groupingunit 50. That is, each advertisement segment denotes a segment storedafter it is determined whether the corresponding advertisement candidatesegment is classified as a direct response advertisement or a normaladvertisement. The advertisement segment stored in the advertisement DB60 may further include a segment identifier (ID), a direct responseadvertisement type identifier, and an instance number, in addition tothe above-described advertisement candidate segment information, as theadvertisement segment information. Further, the advertisement segmentmay further include, as other additional information, region informationrelated to a location at which a character string indicating a directresponse advertisement is displayed and time information related to asection in which an audio signal indicating a direct responseadvertisement is located. In addition, the advertisement segment mayfurther include various types of required meta-information depending onthe circumstances.

The direct response advertisement determination unit 30 functions todetermine whether the advertisement candidate segment is a directresponse advertisement if it is determined by the matching unit 20 thatan advertisement segment matching the advertisement candidate segment isnot present.

As well known in the art, a direct response advertisement (DR ad)denotes an advertisement for attracting viewers' direct responses byexposing information, such as a phone number, an Internet homepageaddress (Uniform Resource Locator: URL), and a price, at a specificlocation of an advertisement video for a predetermined period of time sothat viewers can order a product related to the advertised product orcan request the consultation of the product via, for example, a TVadvertising broadcast. Such a direct response advertisement enables aphone number (for example, 800-323-8370) for product purchasing to beexposed at a specific location of an advertisement video in the case ofthe advertisement video, and the direct response advertisementdetermination unit 30 may determine whether the correspondingadvertisement is a direct response advertisement by checking the exposedphone number.

That is, when the advertisement corresponding to the advertisementcandidate segment is a video signal, the direct response advertisementdetermination unit 30 may determine whether the correspondingadvertisement is a direct response advertisement by checking whether apre-designated character string (for example, a phone number) indicatingthat the advertisement is the direct response advertisement is includedin at least one of the image frames constituting the correspondingadvertisement video signal.

In this case, the pre-designated character string may be, for example, aphone number, an Internet homepage address, a price, or the like. Forexample, as the phone number, only a part of the phone number other thanthe entire phone number may be designated. For example, when a homeshopping company generally uses a telephone exchange number “800”, only“800” may be designated. Further, only “www” indicting an Internetaddress may be pre-designated.

Whether such a character string is included in image frames may bedetermined using character recognition technology known in theconventional technology. The character recognition technology itself isnot a direct object of the present invention and may be implementedusing the conventional technology without change, and thus a detaileddescription thereof will be omitted.

The direct response advertisement determination unit 30 acquires one ormore pieces of region information related to a location at which apre-designated character string is displayed if the pre-designatedcharacter string is included in an image frame.

FIG. 2 is a diagram showing a procedure for acquiring regioninformation.

Referring to FIG. 2, an upper picture is obtained by capturing anadvertisement video provided through a broadcast stream, and one of theimage frames including a preset character string (in FIG. 2, a phonenumber ‘800-323-8370’) in an advertisement video is illustrated. Here,region information related to the location at which the correspondingcharacter string is displayed may be represented by a rectangle, asshown in a lower portion of FIG. 2. The region information may berepresented by R1=[(w11,h11),(w12,h12)] using an upper left vertex (w11,h11) and a lower right vertex (w12, h12) of the rectangle.

As such region information, a plurality of pieces of region informationmay be present in a single advertisement video, that is, anadvertisement candidate segment, and they may be represented byR2=[(w21,h21),(w22,h22)], R3=[(w31,h31),(w32,h32)], . . . , etc. usingupper left vertexes and lower right vertexes, as described above.

Such region information is stored as advertisement segment informationand is used to group direct response advertisements by the directresponse advertisement grouping unit 50, as will be described later.

Meanwhile, the direct response advertisement determination unit 30 maydetermine whether the corresponding advertisement is a direct responseadvertisement even when the advertisement is an audio signal. This maybe performed by checking, using speech recognition technology, whether apre-designated audio signal indicating that the correspondingadvertisement is a direct response advertisement is included inadvertisement audio. In this case, the direct response advertisementdetermination unit 30 may acquire one or more pieces of time informationrelated to a location at which the pre-designated audio signal appears.The speech recognition technology is not a direct object of the presentinvention and may be implemented using conventional technology withoutchange, and thus a detailed description thereof will be omitted.

Referring back to FIG. 1, the direct response advertisement registrationunit 40 will be described.

The direct response advertisement registration unit 40 functions tostore each advertisement candidate segment, determined to be a directresponse advertisement by the direct response advertisementdetermination unit 30, as an advertisement segment in the advertisementDB 60. In this case, the corresponding advertisement candidate segmentis stored as the advertisement segment that is the direct responseadvertisement in the advertisement DB 60.

Together with this, the direct response advertisement registration unit40 may store information about a region in the image frame in which thepre-designated character string, indicating the direct responseadvertisement and acquired by the direct response advertisementdetermination unit 30, is displayed, and information about the characterstring, in the advertisement DB.

Further, information about reliability of the results of characterrecognition used upon checking character string information may beadditionally stored as advertisement segment information for theadvertisement segment in the advertisement DB. As well known in the art,character recognition technology may also detect together thereliability of the results of character recognition. Such reliability isstored together as advertisement segment information in theadvertisement DB 60, and may then be used to set a master segment, whichwill be described later.

Further, when an advertisement is provided as audio, the direct responseadvertisement registration unit 40 may store time information about thetime at which the pre-designated audio signal appears in thecorresponding advertisement candidate segment, and information about thedesignated audio signal in the advertisement DB 60. Furthermore,similarly to the case of video, information about reliability ofchecking audio signal information may be additionally stored, as theadvertisement segment information for the advertisement segment, in theadvertisement DB.

Further, an advertisement candidate segment, not determined to be adirect response advertisement by the direct response advertisementdetermination unit 30, is stored as a normal advertisement segment inthe advertisement DB 60.

Meanwhile, the direct response advertisement registration unit 40 setseach advertisement segment stored in the advertisement DB 60 to themaster segment of the direct response advertisement. The advertisementsegment registered by the direct response advertisement registrationunit 40 is one determined to be a direct response advertisement by thedirect response advertisement determination unit 30, and theadvertisement candidate segment processed by the direct responseadvertisement determination unit 30 is one for which it is determined bythe matching unit 20 that an advertisement segment matching thecorresponding advertisement candidate segment is not present.Accordingly, the advertisement candidate segment stored in theadvertisement DB 60 by the direct response advertisement registrationunit 40 is an advertisement appearing first for the corresponding directresponse advertisement, and thus the advertisement segment is set to themaster segment of the corresponding direct response advertisement. Adetailed operation of a master segment will be described in detail laterwith reference to the direct response advertisement grouping unit 50.

If it is determined by the matching unit 20 that an advertisementsegment matching the advertisement candidate segment is present, and thematching advertisement segment is a direct response advertisement, thedirect response advertisement grouping unit 50 functions to group theadvertisement candidate segment with advertisement segments that aredirect response advertisements stored in the advertisement DB 60.

Here, the term “grouping” denotes the operation of storing advertisementcandidate segments so that they are associated with pre-storedadvertisement segments depending on identicalness to the pre-storedadvertisement segments when storing the advertisement candidate segmentsas advertisement segments in the advertisement DB.

FIG. 3 is a diagram showing the concept of the grouping procedureperformed by the direct response advertisement grouping unit 50.

As described above, there are many cases where, for the sameadvertisement content, direct response advertisements are provided afterthe information thereof, such as phone numbers, Internet homepageaddresses, or prices, has been changed in conformity with thecharacteristics of countries or the areas of each country, forrespective countries or respective areas. The grouping performed by thedirect response advertisement grouping unit 50 provides a method ofclassifying direct response advertisements in consideration of thisfact.

Referring to FIG. 3, the vertical line of FIG. 3 shows that the sameadvertisement content is divided into a master segment and variationsegments, wherein the master segment is an advertisement segmentcorresponding to a direct response advertisement which appears firstamong direct response advertisements or which is designated by otherconditions, and denotes an advertisement segment functioning as areference point for a specific direct response advertisement.

Each variation segment denotes an advertisement segment which hasidenticalness to the master segment from the standpoint of details inthe advertisement content, but has differences in information such as aphone number, an Internet homepage address, or a price from the mastersegment, as described above, and it refers to an advertisement segmenthaving a changed form with respect to the master segment.

Variation segments may be configured to include not only cases wherepieces of information, such as phone numbers, Internet homepageaddresses, or prices, are different from each other, but also caseswhere locations at which these pieces of information appear in anadvertisement video are different from each other, that is, the casedescribed in FIG. 2 in which pieces of region information are differentfrom each other. Further, variation segments may be configured toinclude cases where details of advertisement content are partiallydifferent from each other.

For a specific direct response advertisement, a single master segmentmay be present and a plurality of variation segments may be present.

Meanwhile, the horizontal line of FIG. 3 shows that advertisementsegments having identicalness to the master segment or the correspondingvariation segment are associated with each other, and such advertisementsegments having identicalness to the master or variation segment aredesignated as instance segments in the present invention. Instancesegments present on the horizontal line extending from the mastersegment or each variation segment are advertisement segments havingidenticalness to the master or variation segment on the left side of thecorresponding horizontal line.

FIG. 4 illustrates a master segment and variation segments, and instancesegments of the master and each variation segment, such as thosedescribed in FIG. 3, together with actual screens.

In FIG. 4, the uppermost portion of a vertical line on a left sidedenotes a master segment 400 for a specific direct responseadvertisement, and screens below the master segment respectively denotevariation segments 401 and 402. Screens respectively present on thehorizontal lines extending from the master segment 400 and the variationsegments 401 and 402 denote the instance segments of the correspondingsegments.

It can be seen that, if the master segment 400 is compared with thevariation segments 401 and 402, phone numbers thereof are different fromeach other.

If it is determined by the matching unit 20 that an advertisementsegment matching an advertisement candidate segment is present, and thematching advertisement segment is a direct response advertisement, thedirect response advertisement grouping unit 50 groups the advertisementcandidate segment with the advertisement segments which are directresponse advertisements stored in the advertisement DB 60, using themethod such as that described in FIGS. 3 and 4, classifies theadvertisement candidate segment as a master segment or a variationsegment, and stores the advertisement candidate segment in theadvertisement DB 60, or stores the advertisement candidate segment as aninstance segment of the master or variation segment in the advertisementDB 60.

This procedure will be described in detail below.

First, if an advertisement candidate segment for specific directresponse advertisement A is first determined to be a direct responseadvertisement by the direct response advertisement determination unit30, the advertisement candidate segment is set to a master segment whilebeing stored as an advertisement segment in the advertisement DB 60, asdescribed above.

In this state, if any advertisement candidate segment is generated bythe advertisement candidate segment generation unit 10, the matchingunit 20 determines whether an advertisement segment (direct responseadvertisement) matching the advertisement candidate segment is presentin the advertisement DB 60. If it is determined that the matchingadvertisement segment is present, and the advertisement segment is adirect response advertisement, grouping is performed by the directresponse advertisement grouping unit 50. In this case, the directresponse advertisement grouping unit 50 determines whether theadvertisement candidate segment has identicalness to the master segmentof the matching advertisement segment by comparing those segments witheach other. If the advertisement candidate segment is identical to themaster segment of the matching advertisement segment, the advertisementcandidate segment is set to the instance segment of the master segment.If it is determined that those segments do not have identicalness, theadvertisement candidate segment is set to the variation segment of themaster segment.

If both a master segment and a variation segment are present for aspecific direct response advertisement (advertisement segment), thedirect response advertisement grouping unit 50 sequentially determineswhether the advertisement candidate segment has identicalness to themaster segment and variation segment of the advertisement segmentmatching the advertisement candidate segment.

That is, it is determined first whether the advertisement candidatesegment has identicalness to the master segment by comparing thesegments with each other. If it is determined that the advertisementcandidate segment is identical to an advertisement segment that is themaster segment, the advertisement candidate segment is set to theinstance segment of the master segment.

If it is determined that the advertisement candidate segment is notidentical to the advertisement segment that is the master segment, it isdetermined whether the advertisement candidate segment is identical toan advertisement segment that is the variation segment of the mastersegment by comparing the segments with each other. If it is determinedthat the advertisement candidate segment is identical to theadvertisement segment that is the variation segment, the advertisementcandidate segment is set to the instance segment of the variationsegment, whereas if it is determined that the advertisement candidatesegment is not identical to the advertisement segment that is thevariation segment, the advertisement candidate segment is set to a newvariation segment of the master segment. When there are a plurality ofvariation segments, identicalness to the variation segments issequentially determined, and then the above-described procedure isperformed.

Meanwhile, in the described procedure, the determination ofidenticalness may be performed using the following method.

That is, there can be used a method of comparing an image of a regioncorresponding to region information at which a pre-designated characterstring, indicating a direct response advertisement and included in theadvertisement candidate segment, is located, with an image of a regioncorresponding to region information stored for an advertisement segment(a master segment or each variation segment) which is a comparisontarget may be used. This method may be performed using, for example, amethod of extracting fingerprints of images of the corresponding regionsand comparing the fingerprints with each other.

FIG. 5 is a diagram showing a procedure for determining theidenticalness of region information.

FIGS. 5(a) and (b) illustrate a case where regions corresponding topieces of region information to be compared have a difference of Δh andΔw. In this way, in a region in which a character string is located, aphenomenon corresponding to location changes or screen expansion of Δhand Δw may occur in vertical and horizontal directions, or imageblurring or the like may occur in the corresponding region. In thiscase, it may be determined that identicalness is present. For this, adistance function of regions which are comparison targets is defined bythe following Equation 1, and the presence/non-presence of identicalnessmay be determined based on the results of calculation of the distancefunction.Dist(Rn,Rm)=Dist([(w _(n1) ,h _(n1)),(w _(n2) ,h _(n2))],[(w _(n1) +αω,h_(n1) +Δh),(w _(n2) +Δω,h _(n2) +Δh])  [Equation 1]

In this case, Rn and Rm denote respective regions, and w and h denoteupper left coordinate values and lower right coordinate values of thecorresponding region as described in FIG. 2. Δh and Δw denotedifferences in vertical height and lateral length between the regions tobe compared with each other. When fingerprints are compared with eachother, the distance function may be a difference in the results of acomparison between the fingerprints of two comparison regions. If theresults of calculation of the distance function are compared with apreset threshold, and a resulting value is equal to or less than thethreshold, it is determined that identicalness is present, whereas ifthe resulting value is greater than the threshold, it is determined thatidenticalness is not present.

As another method, a method of comparing character string informationobtained by recognizing a pre-designated character string, indicating adirect response advertisement and included in an advertisement candidatesegment, with the character string information of an advertisementsegment which is a comparison target may be used.

Meanwhile, information about a region in which a pre-designatedcharacter string, indicating a direct response advertisement andincluded in an advertisement candidate segment, is located may beobtained, and this region information may be compared with regioninformation stored for an advertisement segment (a master segment or avariation segment) which is a comparison target, and then thepresence/non-presence of identicalness may be determined. In this case,whether the pieces of region information are identical to each other maybe determined in such a way as to determine that pieces of regioninformation have identicalness not only in a case where coordinatevalues are completely identical, as shown in FIG. 2, but also in a casewhere the coordinate values have a difference within the allowable rangeof a predetermined threshold.

If the above procedure is performed, the direct response advertisementgrouping unit 50 groups each advertisement candidate segment based ondetermination of whether identicalness to advertisement segments (mastersegment and variation segments) stored in the advertisement DB 60 ispresent, and stores the corresponding advertisement candidate segment asan advertisement segment in the advertisement DB 60 so that theadvertisement segment has the structure of FIG. 7, which will bedescribed later.

FIG. 6 is a diagram illustrating a procedure for setting a variationsegment.

In FIG. 6, when (a) illustrates an advertisement segment which is apre-stored master segment, and (b) illustrates an advertisementcandidate segment having information identical to that of (a) except fora phone number, in comparison with (a), if segments (b) and (a) arecompared with each other, in detail, if images corresponding to piecesof region information, or pieces of information of correspondingcharacter strings are compared with each other, segment (b) is groupedas a variation segment of segment (a).

Meanwhile, when, compared to (a), (c) illustrates an advertisementcandidate segment having information identical to that of (a) except fora phone number included in region information, if the segment (c) iscompared with segment (a), the segment (c) is classified as a variationsegment of segment (a).

When (c) and (b) are compared with each other, (c) illustrates a segmentin which a phone number included in region information is identical tothat of (b) and which has a location differing from that of (b), andthus the segment (c) may be an instance segment of segment (b).

Meanwhile, when an instance segment is set for a master segment, one ofthe master segment and the instance segment, which has higherreliability information, may be set to the master segment by comparingpieces of reliability information obtained in the character recognitionmethod used to check character string information, as described above.This is the same even for an instance segment of the variation segment.

Further, after grouping has been performed by the direct responseadvertisement grouping unit 50, grouping results may be displayed viathe user interface of a manager's computer, and the manager may checkthe corresponding details and finally approve or change grouping. Inaddition, results performed by the direct response advertisementdetermination unit 30 and the direct response advertisement registrationunit 40 may be displayed on the manager computer, and the manager mayalso approve or change the corresponding details.

Further, although not shown in the drawings, the system 100 may furtherinclude a statistics unit, so that various types of information abouteach direct response advertisement are divided into pieces ofinformation of a master segment and variation segments and are provided.Forms in which specific direct response advertisements are changed andprovided via broadcasts may be obtained, together with various types ofadditional information, by the statistics unit.

FIG. 7 is a diagram showing an example of advertisement segmentinformation stored in the advertisement DB 60 used in theabove-described system 100.

Referring to FIG. 7, advertisement segment information may include asegment identifier, a direct response advertisement separator, aninstance number, channel_broadcast start time information, sectioninformation, fingerprint information, source data information, andadditional information.

A segment identifier is an identifier uniquely assigned to identify eachadvertisement segment.

A direct response advertisement separator is an identifier required toidentify whether, as described above, the corresponding segment is amaster segment or a variation segment, wherein M in FIG. 7 denotes amaster segment and V denotes a variation segment.

The direct response advertisement separators in first and fourth rows ofFIG. 7 are identical to each other, wherein, of these segments, a mastersegment may be identified by an instance number. That is, a segmenthaving an instance number of 0 is a master segment, and a segment havingan instance number of 1 denotes an instance segment registered first forthe master segment. In the direct response advertisement separators ofFIG. 7, numbers appearing after M or V denote segment identifiers.

Meanwhile, all direct response advertisement separators in the secondrow and the fifth row of FIG. 7 are V, and segment identifiers appearingafter V are 990018, and thus the corresponding segments denote variationsegments corresponding to the segment identifier of 990018. In thiscase, a number appearing after 990018 denotes the sequential position ofthe corresponding variation segment, wherein 1 indicates a firstvariation segment and 2 indicates a second variation segment.

A channel_broadcast start time denotes the name of a channel throughwhich the corresponding advertisement segment is broadcasted, andinformation about the start time at which an advertisement isbroadcasted through the corresponding channel, and section informationdenotes information about a time interval, during which theadvertisement is played, from the broadcast start time.

Fingerprint information denotes fingerprint information extracted fromat least a portion of an advertisement video or audio of theadvertisement segment, and source data denotes original digital data ofthe corresponding advertisement video or audio itself.

As described above, additional information may include regioninformation represented by, for example, lower left coordinate valuesand upper right coordinate values, the determined character stringinformation when a character string indicating a direct responseadvertisement is determined, and other information.

FIG. 8 is a flowchart showing an embodiment of a method performed by thesystem 100 for detecting and classifying direct response advertisementsaccording to the present invention, which has been described in detailwith reference to FIGS. 1 to 7.

Referring to FIG. 8, the advertisement candidate segment generation unit10 generates an advertisement candidate segment for an advertisementsection detected from a broadcast stream (S100).

Next, the matching unit 20 determines whether the advertisementcandidate segment generated at S100 matches each advertisement segmentstored in the advertisement DB (S110).

If it is determined at S110 by the matching unit 20 that anadvertisement segment matching the advertisement candidate segment isnot present, the direct response advertisement determination unit 30determines whether the advertisement candidate segment is a directresponse advertisement (S120).

If it is determined at S130 that the advertisement candidate segment isthe direct response advertisement, the direct response advertisementregistration unit 40 stores the corresponding advertisement candidatesegment, as an advertisement segment that is the direct responseadvertisement, in the advertisement DB (S140).

Meanwhile, if it is determined at S110 by the matching unit 20 that anadvertisement segment matching the advertisement candidate segment ispresent, and the matching advertisement segment is a direct responseadvertisement, the direct response advertisement grouping unit 50 groupsthe advertisement candidate segment with advertisement segments that aredirect response advertisements stored in the advertisement DB 60 (S150).

FIG. 9 is a detailed flowchart showing the grouping procedure at S150 ofFIG. 8.

The detailed flowchart of FIG. 9 illustrates a procedure for setting amaster segment and a variation segment and a procedure for settinginstance segments for the master or variation segment in the groupingprocedure, and is based on the details described with reference to FIGS.3 to 6.

First, among advertisement segments matching an advertisement candidatesegment, a master segment is determined from, for example, theadvertisement DB 60 configured as shown in FIG. 7 (S151).

Once the master segment is determined, it is determined whether theadvertisement candidate segment has identicalness to the master segment(S152).

If it is determined that identicalness to the master segment is present,the advertisement candidate segment is registered and stored, as theinstance segment of the corresponding master segment, in theadvertisement DB 60 (S153).

If it is determined that identicalness to the master segment is notpresent, it is determined whether a variation segment is present for themaster segment, based on the advertisement DB 60 such as that in FIG. 7(S154). If it is determined that a variation segment is not present, theadvertisement candidate segment is registered as a new variation segmentof the master segment (S155).

If it is determined that the variation segment is present, it isdetermined whether the advertisement candidate segment has identicalnessto the variation segment (S156). If it is determined that identicalnessto the variation segment is present, the advertisement candidate segmentis stored as an instance segment of the variation segment in theadvertisement DB 60 (S157). If it is determined that identicalness tothe variation segment is not present, the advertisement candidatesegment is registered as a new variation segment of the master segment(S155).

Although the preferred embodiments of the present invention have beendisclosed for illustrative purposes, those skilled in the art willappreciate that the present invention is not limited to thoseembodiments and various changes and modifications are possible.

The invention claimed is:
 1. A method comprising: receiving, by areceiver, a broadcast stream; generating an advertisement candidatesegment for an advertisement section detected from the receivedbroadcast stream; comparing the advertisement candidate segment to eachadvertisement segment stored in an advertisement database (DB), whereincomparing the advertisement candidate segment to each advertisementsegment stored in the advertisement DB comprises comparing a fingerprintof at least a portion of the advertisement candidate segment to afingerprint of at least a portion of each advertisement segment storedin the advertisement DB; based on the comparing, making a firstdetermination that the advertisement candidate segment does not matchany advertisement segment stored in the advertisement DB, wherein makingthe first determination comprises determining that the advertisementcandidate segment lacks a threshold extent of similarity with each ofthe advertisement segments stored in the advertisement DB; making asecond determination that the advertisement candidate segment is adirect response advertisement; and based on making the first and seconddeterminations, storing into the advertisement DB, (i) the advertisementcandidate segment, and (ii) information associated with theadvertisement candidate segment, wherein the information indicates thatthe advertisement candidate segment is a master segment of a directresponse advertisement.
 2. The method of claim 1, wherein the receivedbroadcast stream includes an advertisement section, and wherein theadvertisement candidate segment is part of the advertisement section. 3.The method of claim 1, wherein making the second determination that theadvertisement candidate segment is a direct response advertisementcomprises determining that a pre-designated character string is includedin at least one frame of the advertisement candidate segment.
 4. Themethod of claim 3, wherein determining that a pre-designated characterstring is included in at least one frame of the advertisement candidatesegment comprises using a character recognition technique to determinethat a pre-designated character string is included in at least one frameof the advertisement candidate segment.
 5. The method of claim 1,wherein making the second determination that the advertisement candidatesegment is a direct response advertisement comprises determining that aphone number is included in at least one frame of the advertisementcandidate segment.
 6. The method of claim 5, wherein determining that aphone number is included in at least one frame of the advertisementcandidate segment comprises using a character recognition technique todetermine that a phone number is included in at least one frame of theadvertisement candidate segment.
 7. A method comprising: receiving, by areceiver, a broadcast stream generating an advertisement candidatesegment for an advertisement section detected from the receivedbroadcast stream; comparing the advertisement candidate segment to eachadvertisement segment stored in an advertisement database (DB), whereincomparing the advertisement candidate segment to each advertisementsegment stored in the advertisement DB comprises comparing a fingerprintof at least a portion of the advertisement candidate segment to afingerprint of at least a portion of each advertisement segment storedin the advertisement DB; making a first determination that theadvertisement candidate segment does not match any advertisement segmentstored in the advertisement DB, wherein making the first determinationcomprises determining that the advertisement candidate segment lacks athreshold extent of similarity with each of the advertisement segmentsstored in the advertisement DB; making a second determination that theadvertisement candidate segment is a direct response advertisement; andstoring into the advertisement DB, (i) the advertisement candidatesegment, and (ii) information associated with the advertisementcandidate segment, wherein the information indicates that theadvertisement candidate segment is a master segment of a direct responseadvertisement.
 8. The method of claim 7, wherein the received broadcaststream includes an advertisement section, and wherein the advertisementcandidate segment is part of the advertisement section.
 9. The method ofclaim 7, wherein making the second determination that the advertisementcandidate segment is a direct response advertisement comprisesdetermining that a pre-designated character string is included in atleast one frame of the advertisement candidate segment.
 10. The methodof claim 7, wherein making the second determination that theadvertisement candidate segment is a direct response advertisementcomprises determining that a phone number is included in at least oneframe of the advertisement candidate segment.